This invention relates to an image processing system for simulating depth of field in a captured image.
A shallow depth of field can be produced in an image by using a camera lens having a large aperture (typically expressed as a small f-number). Compact digital cameras and digital cameras integrated into smartphones and tablets are, however, not typically equipped with a lens system which is capable of achieving a sufficiently large aperture to produce a shallow depth of field.
Various techniques are described in the prior art for simulating a depth of field effect in a digital camera which is not equipped with a lens system capable of a large enough aperture size to physically produce a shallow depth of field. These techniques rely on digitally post-processing captured images to add a depth of field effect.
For example, U.S. Pat. No. 9,025,874 uses information generated by an autofocus system to form a measure of the relative focal depth of regions defined for a captured image. An amount of blur is then applied to each region according to the difference between the focal depth of that region and the focal distance selected for the captured image by the autofocus system for the region of interest (i.e. the object(s) in the image which it is desired are in focus). Chinese Patent 101764925 describes a similar system. This approach suffers from several problems. If only a limited number of focus regions are defined, each region can itself span a range of focal depths meaning that any blur applied to that region will not be accurate for all of the features in the region. If a large number of focus regions are defined, the processing requirements would be very substantial—although in practice the limiting factor is likely to be the resolution of focus information from the autofocus system, which will normally only provide focus information for a limited number of zones (e.g. no more than 16).
Furthermore, in order to accurately determine a measure of the focal distance for a number of focus regions it is necessary to capture several frames at a range of focal distances. In order to provide a reasonable depth of field effect, such conventional systems will therefore suffer from high latency, substantial image processing requirements and high power consumption. Operating such systems using focus information determined over only a limited number of frames will lead to poor performance and an unrealistic depth of field effect.
Another approach taken in the prior art in order to simulate a depth of field effect is to require the user to indicate a region of the image which is to be in focus. This technique is described in US Patent Application 2009/0160966 in which a blur effect is applied to all regions outside a region selected by the user. The selected region has a predefined shape (e.g. an ellipse). This approach provides an unrealistic depth of field since the same amount of blur is applied to all regions outside the selected region, irrespective of their true focal depth. In the vast majority of cases, the predefined shape of the region which is to be in focus is unlikely to correspond well to the features in the image. The approach only ever provides a rough approximation of depth of field in simple cases where a central object is well spaced in front of a homogenous background (e.g. when taking a photo of a person in front of a landscape).